Microchips convolucionadores AER para procesado asíncrono neocortical de información sensorial visual codificada en eventos

Tesis descargada desde TeseoEn este trabajo, se presentan dos versiones diferentes de microchips convolucionadores completamente digitales basados en el protocolo AER para sistemas de procesamiento visual basados en eventos. Estos chips constituyen la unidad básica para construir sistemas complejos multicapa a partir de la interconexión en serie y en paralelo de diferentes muestras de los mismos. Cada uno de ellos permite re alizar convoluciones con kernel programable de un tamaño máximo de 32*32.La primera versión de chip Conv1 opera sobre un array de píxeles de tamaño 32*32, aunque está diseñado para poder construir sistemas equivalentes de mayor resolución conectando varias muestras en paralelo. La segunda versión Conv2 cuenta con un tamaño 4 veces mayor, 64*64 píxeles, y además implementa la funcionalidad multikernel. Esta funcionalidad permite programar varios kernels diferentes dentro de un mismo chip (hasta 32) para que éste pueda recibir eventos de varios chips diferentes de una capa anterior, y aplicarle a cada uno de ellos un kernel diferente en función de su origen. Esto está especialmente indicado para facilitar la implementación de sistemas multicapa, a imitación de la corteza cerebral, y siguiendo las estructuras típicas del paradigma conocido como “Convolutional Neural Networks”.En el presente documento se describen detalladamente las arquitecturas de cada una de las dos versiones propuestas de chips de convolución, así como algunos resultados experimentales obtenidos. El documento está estructurado de la siguiente forma. En primer lugar, en el Capítulo 2 se presentan las ventajas de los sistemas de procesamiento visual basados en eventos, frente a los tradicionales sistemas basados en fotogramas. Este capítulo incluye también una descripción del protocolo AER, imprescindible para construir sistemas de procesamiento por eventos. El Capítulo 3 hace un repaso sobre las estructuras multicapa bioinspiradas de procesamiento de imagen, justificando el uso de la operación de convolución como unidad básica de este tipo de sistemas, describiendo también la arquitectura propuesta para los chips de convolución basados en AER. En el Capítulo 4 se describe en detalle el píxel de convolución como base de los chips propuestos, analizando las dos versiones diferentes, mientras que en el Capítulo 5 se detallan el resto de circuitos periféricos incluidos en los chips de convolución. El Capítulo 6 muestra exhaustivos resultados experimentales obtenidos a partir de las dos versiones Conv1 y Conv2. Por último, el Capítulo 7 presenta las conclusiones

Compact Functional Testing for Neuromorphic Computing Circuits

We address the problem of testing artificial intelligence (AI) hardware accelerators implementing spiking neural networks (SNNs). We define a metric to quickly rank available samples for training and testing based on their fault detection capability. The metric measures the interclass spike count difference of a sample for the fault-free design. In particular, each sample is assigned a score equal to the spike count difference between the first two top classes. The hypothesis is that samples with small scores achieve high fault coverage because they are prone to misclassification, i.e., a small perturbation in the network due to a fault will result in these samples being misclassified with high probability. We show that the proposed metric correlates with the per-sample fault coverage and that retaining a set of high-ranked samples in the order of ten achieves near-perfect fault coverage for critical faults that affect the SNN accuracy. The proposed test generation approach is demonstrated on two SNNs modeled in Python and on actual neuromorphic hardware. We discuss fault modeling and perform an analysis to reduce the fault space so as to speed up test generation time. © 1982-2012 IEEE

An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors

Event-Driven vision sensing is a new way of sensing visual reality in a frame-free manner. This is, the vision sensor (camera) is not capturing a sequence of still frames, as in conventional video and computer vision systems. In Event-Driven sensors each pixel autonomously and asynchronously decides when to send its address out. This way, the sensor output is a continuous stream of address events representing reality dynamically continuously and without constraining to frames. In this paper we present an Event-Driven Convolution Module for computing 2D convolutions on such event streams. The Convolution Module has been designed to assemble many of them for building modular and hierarchical Convolutional Neural Networks for robust shape and pose invariant object recognition. The Convolution Module has multi-kernel capability. This is, it will select the convolution kernel depending on the origin of the event. A proof-of-concept test prototype has been fabricated in a 0.35 m CMOS process and extensive experimental results are provided. The Convolution Processor has also been combined with an Event-Driven Dynamic Vision Sensor (DVS) for high-speed recognition examples. The chip can discriminate propellers rotating at 2 k revolutions per second, detect symbols on a 52 card deck when browsing all cards in 410 ms, or detect and follow the center of a phosphor oscilloscope trace rotating at 5 KHz.Unión Europea 216777 (NABAB)Ministerio de Ciencia e Innovación TEC2009-10639-C04-0

Event-Driven Stereo Visual Tracking Algorithm to Solve Object Occlusion

Object tracking is a major problem for many computer vision applications, but it continues to be computationally expensive. The use of bio-inspired neuromorphic event-driven dynamic vision sensors (DVSs) has heralded new methods for vision processing, exploiting reduced amount of data and very precise timing resolutions. Previous studies have shown these neural spiking sensors to be well suited to implementing singlesensor object tracking systems, although they experience difficulties when solving ambiguities caused by object occlusion. DVSs have also performed well in 3-D reconstruction in which event matching techniques are applied in stereo setups. In this paper, we propose a new event-driven stereo object tracking algorithm that simultaneously integrates 3-D reconstruction and cluster tracking, introducing feedback information in both tasks to improve their respective performances. This algorithm, inspired by human vision, identifies objects and learns their position and size in order to solve ambiguities. This strategy has been validated in four different experiments where the 3-D positions of two objects were tracked in a stereo setup even when occlusion occurred. The objects studied in the experiments were: 1) two swinging pens, the distance between which during movement was measured with an error of less than 0.5%; 2) a pen and a box, to confirm the correctness of the results obtained with a more complex object; 3) two straws attached to a fan and rotating at 6 revolutions per second, to demonstrate the high-speed capabilities of this approach; and 4) two people walking in a real-world environment.Ministerio de Economía y Competitividad TEC2012-37868-C04-01Ministerio de Economía y Competitividad TEC2015-63884-C2-1-PJunta de Andalucía TIC-609

A Configurable Event-Driven Convolutional Node with Rate Saturation Mechanism for Modular ConvNet Systems Implementation

Convolutional Neural Networks (ConvNets) are a particular type of neural network often used for many applications like image recognition, video analysis or natural language processing. They are inspired by the human brain, following a specific organization of the connectivity pattern between layers of neurons known as receptive field. These networks have been traditionally implemented in software, but they are becoming more computationally expensive as they scale up, having limitations for real-time processing of high-speed stimuli. On the other hand, hardware implementations show difficulties to be used for different applications, due to their reduced flexibility. In this paper, we propose a fully configurable event-driven convolutional node with rate saturation mechanism that can be used to implement arbitrary ConvNets on FPGAs. This node includes a convolutional processing unit and a routing element which allows to build large 2D arrays where any multilayer structure can be implemented. The rate saturation mechanism emulates the refractory behavior in biological neurons, guaranteeing a minimum separation in time between consecutive events. A 4-layer ConvNet with 22 convolutional nodes trained for poker card symbol recognition has been implemented in a Spartan6 FPGA. This network has been tested with a stimulus where 40 poker cards were observed by a Dynamic Vision Sensor (DVS) in 1 s time. Different slow-down factors were applied to characterize the behavior of the system for high speed processing. For slow stimulus play-back, a 96% recognition rate is obtained with a power consumption of 0.85mW. At maximum play-back speed, a traffic control mechanism downsamples the input stimulus, obtaining a recognition rate above 63% when less than 20% of the input events are processed, demonstrating the robustness of the networkEuropean Union 644096, 687299Gobierno de España TEC2016-77785- P, TEC2015-63884-C2-1-PJunta de Andalucía TIC-6091, TICP120

Fully Digital AER Convolution Chip for Vision Processing

We present a neuromorphic fully digital convolution microchip for Address Event Representation (AER) spike-based processing systems. This microchip computes 2-D convolutions with a programmable kernel in real time. It operates on a pixel array of size 32 x 32, and the kernel is programmable and can be of arbitrary shape and size up to 32 x 32 pixels. The chip receives and generates data in AER format, which is asynchronous and digital. The paper describes the architecture of the chip, the test setup, and experimental results obtained from a fabricated prototype.European Union IST-2001-34124 (CAVIAR)Comisión Interministerial de Ciencia y Tecnología TIC-2003-08164-C03-01Ministerio de Educación y Ciencia TEC2006-11730-C03-01Junta de Andalucía P06-TIC-0141

On the use of orientation filters for 3D reconstruction in event-driven stereo vision

The recently developed Dynamic Vision Sensors (DVS) sense visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, therefore increasing the number of constraints applied to the matching algorithm. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.ERANET PRI-PIMCHI- 2011-0768Ministerio de Economía y Competitividad TEC2009-10639-C04-01, TEC2012-37868- C04-01Junta de Andalucía TIC-609

Event-driven stereo vision with orientation filters

The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.European Union PRI-PIMCHI-2011-0768Ministerio de Economía y Competitividad TEC2009-10639-C04-01Ministerio de Economía y Competitividad TEC2012-37868-C04-01Junta de Andalucía TIC-609

Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition

Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.Ministerio de Educación y Ciencia TEC-2006-11730-C03-01Junta de Andalucía P06-TIC-01417European Union IST-2001-34124, 21677

Event-Driven Configurable Module with Refractory Mechanism for ConvNets on FPGA

We have developed a fully configurable event-driven convolutional module with refractory period mechanism that can be used to implement arbitrary Convolutional Neural Networks (ConvNets) on FPGAs following a 2D array structure. Using this module, we have implemented in a Spartan6 FPGA a 4-layer ConvNet with 22 convolutional modules trained for poker card symbol recognition. It has been tested with a stimulus where 40 poker cards were observed by a Dynamic Vision Sensor (DVS) in 1s time. A traffic control mechanism is implemented to downsample high speed input stimuli while keeping spatio-temporal correlation. For slow stimulus play back, a 96% recognition rate is achieved with a power consumption of 0.85mW. At maximum play back speed, the recognition rate is still above 63% when less than 20% of the input events are processed.European Union's Horizon 2020 644096 (ECOMODEEuropean Union's Horizon 2020 No 687299 NeuRAMMinisterio de Ciencia, Innovación y Universidades TEC2015- 63884-C2-1-PJunta de Andalucía TIC-609